The present invention relates generally to computer systems, and more particularly to a system and method to enable smooth mixing and integration of a selected foreground image with an unassociated background image by determining mixture-based opacity values along a contour of the foreground image.
Digital images and related image processing have created a profound impact in all aspects of modern society. From blockbuster movies to the classroom, from business presentations to the daily weather report, digital images affect and influence peoplexe2x80x94dozens, perhaps, hundreds of times per day. For example, with the advent of communications technologies such as the Internet, business persons, students, researchers and ordinary citizens routinely transmit and receive digital images in the normal course of daily activity. Thus, since digital images have become a staple in modern society, ever changing and more sophisticated requirements for image processing are consistently challenging systems designers and architects.
One such challenge relating to image processing is associated with seamlessly combining portions of a first image with portions of a second image. For example, when observing the nightly weather report, the image of the weather person is often interposed with a plurality of background screens depicting various weather patterns. In the case of high-tech movie special effects and/or other computer generated images, an object, person and/or scene portion, known as the foreground, is often digitally extracted from an associated background scene and placed into a second scene having entirely different backgrounds from the first scene. In order to accomplish this type of image extraction and migration however, it is important that remnants or artifacts of the first scene do not appear in the second scene, and that the extracted object, person or scene portion seamlessly migrate into the second scene. In other words, the migrated foreground image should appear without rough edges and appear as though the new background scene was the original background. Unfortunately, conventional image processing systems many times do not effectively achieve these goals.
One such conventional model for extracting foreground image regions from an associated background region relates to utilizing an artificial/engineered and/or clean background (e.g., blue screen), and extracting the foreground image from the artificial background via a background subtraction technique. This may be achieved by processing image pixels and determining whether a threshold level has been attained after subtracting known background pixel values from each pixel value in the image. For example, according to the conventional model, given the known background pixel values of the artificial background, the known background pixel value is subtracted from each pixel in the image and compared to a predetermined threshold. If the result of the subtraction is below the predetermined threshold, the pixel is assumed to be a background pixel and thus not assigned a value in the extracted image. If the result is above the predetermined threshold, the pixel is assumed to be a foreground pixel and thus retains its original value. Unfortunately, background subtraction and other extraction techniques may not cleanly separate foreground regions of the image and enable smooth placement of the foreground onto a new background. Some of the problems associated with these techniques relate to xe2x80x9cbluescreenxe2x80x9d reflections in the extracted foreground, potential xe2x80x9cholesxe2x80x9d in the foreground, wherein the values of the foreground and background are mixed, jagged edges along the contours of the extracted image, and xe2x80x9cbleedingxe2x80x9d of the previous background into the new background. Moreover, providing artificial clean backgrounds is often not possible and/or difficult to achieve.
In view of the above problems associated with conventional image processing systems, there is a need for a system and/or methodology to facilitate precise extraction of a foreground region of an image from an associated background region, and to enable smooth migration of the foreground region to other background regions.
The following presents a simplified summary of the invention in order to provide a basic understanding of some aspects of the invention. This summary is not an extensive overview of the invention. It is intended to neither identify key or critical elements of the invention nor delineate the scope of the invention. Its sole purpose is to present some concepts of the invention in a simplified form as a prelude to the more detailed description that is presented later.
The present invention relates to a system and methodology for enabling portions of image regions to be extracted from adjacent background regions via a mixture-based opacity determination near the contours of the extracted image region. In this manner, blurring effects and unwanted background portions are mitigated in the extracted image region. An extraction tool may be employed to return a set of pixels associated with the contour of an image foreground region. In accordance with the present invention, the set of pixels are dilated to include regions of blur relating to the contour. Each pixel in the dilated set is analyzed wherein two values are computed to separate the image foreground region from its background. These values may be determined as a foreground value and an alpha value. The dilation process may include a user selecting a width of dilation for the set of pixels. Once selected, a standard morphological algorithm of dilation may be applied to xe2x80x9cwidenxe2x80x9d the set. It is to be appreciated that other techniques for performing dilation may be employed, and that some of these techniques may vary the degree of dilation for each pixel in the contour set.
Generally, because of image blurring, image pixels near a foreground/background boundary result in a mixture of both the foreground color and the background color. This mixture may be generally modeled linearly as follows: I=xcex1F+(1xe2x88x92xcex1)B, wherein I represents an actual pixel value, F is the foreground color, B is the background color, and xcex1 is the alpha value which represents a mixing ratio of foreground and background colors. Since an I value may be known from a given image, xcex1, F and B are to be determined.
Related to the image smoothness, it may be expected that a nearby pixel from the foreground side of a contour may have a value similar to the value that mixed with the background to generate 1. Thus, according to one aspect of the invention, an F value for a given contour pixel may be borrowed from the nearest foreground pixel, defined as the pixel that is closet to the pixel in question, which is not in the dilated pixel set. It is to be appreciated that there are many other possibilities for determining the F value. For example, F values may be determined via (1) an average of several nearby foreground pixels, (2) a stochastically selected, weighted average of nearby pixels, (3) a pixel generated by a texture reconstruction process applied to nearby foreground pixels, as well as (4) a mixture of an I value and the pixel value generated by any of (1-3)) above. Similarly, a B value may be borrowed for a given contour pixel from the nearest background pixel. Utilizing the determined values for F and B, and knowing I, the value of alpha (xcex1), which is the mixing ratio of foreground and background colors may be solved approximately, for example via the following equation:   α  =                    (                  I          -          B                )            ·              (                  F          -          B                )                            "LeftDoubleBracketingBar"                  F          -          B                "RightDoubleBracketingBar"            2      
After determining the mixing ratio, image pixels within the dilated region may be extracted and mixed with a subsequent background region. Thus, improved image extraction from a blurred background region may be achieved over conventional systems by determining alpha, F and B.
According to one aspect of the present invention, a system is provided for selecting a foreground region of an image, given a set of pixels defining the boundary of the foreground region of the image. The system includes a first component to dilate the pixel set and a second component to determine a foreground value (F) and to determine an opacity value (a) based on estimated foreground and background values for each pixel of the dilated set to facilitate a separation of the region from background portions of the image.
According to another aspect of the present invention, a method is provided for integrating extracted images. The method includes: selecting a set of contour pixels defining a foreground region of an image; dilating a set of adjacent pixels for each contour pixel; and utilizing a mixture-based computation for determining an opacity value (a) for each contour pixel and each pixel within the dilated set of adjacent pixels.
In accordance with another aspect of the present invention, a system is provided for extracting images. The system includes: means for selecting a set of contour pixels defining a foreground region of an image; means for dilating a set of adjacent pixels for each contour pixel. The system utilizes a mixture-based computation to determine an opacity value (a) for each contour pixel and each pixel within the dilated set of adjacent pixels.
According to another aspect of the present invention, a signal is provided for integrating extracted images. The signal includes communicating information associated with an image. A first component selects a set of contour pixels defining a foreground region of the image, and dilating a set of adjacent pixels for each contour pixel via the signal. A second component utilizes a mixture-based computation for determining an opacity value (a) via the signal for each contour pixel and each pixel within the dilated set of adjacent pixels to smoothly mix the foreground region of the image with a subsequent background region associated with another image. The signal is communicated over at least one of a network system and a wireless system.
The following description and the annexed drawings set forth in detail certain illustrative aspects of the invention. These aspects are indicative, however, of but a few of the various ways in which the principles of the invention may be employed and the present invention is intended to include all such aspects and their equivalents. Other advantages and novel features of the invention will become apparent from the following detailed description of the invention when considered in conjunction with the drawings.